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Abstract 

Maximum entropy principle does not seem to distinguish between the use of 
Tsallis and Renyi entropies as either of them may be used to derive similar 
power-law distributions. In this paper, we address the question whether 
the Renyi entropy is equally suitable to describe those systems with power- 
law behaviour, where the use of the Tsallis entropy is relevant. We discuss 
a specific class of dynamical systems, namely, one dimensional dissipative 
maps at chaos threshold and make our study from two aspects: i) power- 
law sensitivity to the initial conditions and the rate of entropy increase, 
ii) generalized bit cumulants. We present evidence that the Tsallis entropy 
is the more appropriate entropic form for such studies as opposed to Renyi 
form. 

PA CS Number (s): 05.45.-a, 05.20.-y, 05.70. Ce 
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I. INTRODUCTION 



As it is evident, an enormous variety of systems in Nature fall into the domain of valid- 
ity of Boltzmann-Gibbs (BG) thermostatistics. On the other hand, it is also well-known 
since long that a variety of anomalous systems exists for which the powerful BG formalism 
exhibits serious difficulties. Some of the examples for such anomalous cases could be the 
long-ranged interacting systems M, two-dimensional turbulence nonmarkovian pro- 
cesses [[|, granular matter M, cosmology 0, high energy collisions ||, among others. In 
order to handle some of these anomalous systems, an attempt has been performed in 1988 
0, which is based on the generalization of the standard BG formalism by postulating a 
nonextensive entropy (Tsallis entropy) of the form 

S q = ^= lP * (qeTZ) , (1) 

and it recovers the standard BG entropy S\ = — J2^LiPi^ n Pi m the q — > 1 limit. This 
generalization is usually called in the literature as Nonextensive thermostatistics or Tsallis 
thermostatistics (TT) ||, and it has been (still continues to be!) a matter of intensive 
studies during the past decade both from the point of theoretical foundations of the 
formalism and its applications to various physical systems. The apparent success of TT 
gave rise to an increase of the studies with new entropy definitions ]IU| . At this point, 
we would like to mention a trivial but important issue: TT by no means intends to cover 
all of the anomalous physical systems where BG statistics seems to fail. It appears to 
be useful only for a large class of cases where power-law behaviour is observed. In this 
sense, alternate forms of entropy could of course be useful for other subexponential cases 
and therefore other possible generalizations of the standard formalism could always be 
projected using different entropic forms. One of these attempts, based on the Renyi 
entropy, has become popular nowadays |11 , 12[ . The form of this entropy is [|1^ 



s r = In^P? (2) 
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It is an extensive quantity (unlike Tsallis entropy), concave for < q < 1 and it recovers 
the usual BG entropy as a special case when q — > 1. It is worth mentioning that the Renyi 
entropy has already been used in the multifractal theory [|14]]. On the other hand, the 
efforts of establishing a thermostatistics from this entropy seem to be originated from the 
fact that the maximum entropy principle yields the same form of distribution function (of 
power-law type) for both Tsallis and Renyi entropies since they are monotonic functions 
of each other. At this level, there is no a priori reason to choose one of these entropies as 
the correct entropic form for the systems which exhibit power-law type behaviour. This is 
not the case for the standard BG entropy, namely, everybody knows that any monotonic 
function of the BG entropy also gives the same distribution function of exponential type, 
but still the correct description of entropy is of course BG. This unclear situation of the 
generalized formalism seems to force some authors, for example Arimitsu and Arimitsu, 
to use unclear statements like "An analytical expression of probability density function of 
velocity fluctuation is derived with the help of the statistics based on the Tsallis entropy 
or the Renyi entropy" [|T5|]. 

From the above discussion, a straightforward question arises naturally: Which of these 
two entropic forms is the correct definition for systems exhibiting power-law behaviour ? 
Our motivation in this paper is to try to provide an answer (at least to make the first 
step) to this important and intriguing question. As already seen, since maximum entropy 
principle cannot provide an answer, in order to accomplish this task, we shall focus on 
two different issues and make use of the results coming from them. These issues are 
i) power-law sensitivity to the initial conditions and entropy increase rates, ii) generalized 
bit cumulant theory. 
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II. POWER-LAW SENSITIVITY TO THE INITIAL CONDITIONS AND 

ENTROPY INCREASE RATES 



As is well-known from the theory of dynamical systems (see for example [16,17]), for 
one-dimensional dissipative systems, it is possible to introduce a sensitivity function of 
type 

/ x , Ax(t) , , 

£(t) = lim —44 , 3 

where Ax(0) and Ax(t) are discrepancies of the initial conditions at times and t re- 
spectively, and it satisfies the differential equation d£/dt = Ai£, where Ai is the standard 
Lyapunov exponent, thus £(t) is of exponential type (£(£) = e Xlt ). Consequently, when 
Ai > 0, the system is strongly sensitive to the initial conditions, whereas it is strongly 
insensitive if X\ < 0. On the other hand, there is an infinite number of points for which 
Ai = 0. This case is called the marginal case and no further analysis of this case is pos- 



sible within the standard theory. It has been conjectured recently that |18j whenever Ai 
vanishes, the sensitivity function becomes of power-law type 

ew = [i + (i-?)v] l/a " 9) . ( 4 ) 

which is the solution of the differential equation d£/dt = X q ^ q . This new definition of the 
sensitivity function recovers the standard exponential one in the q — > 1 limit and here, 
X q is the generalized Lyapunov exponent and it scales with time inversely as Ai does, but 
this time exhibits a power-law behaviour. Consequently, if X q > and q < 1 (X q < and 
q > 1), then the system is weakly sensitive (insensitive) to the initial conditions. Most 
important case which is in the domain of this scenario is of course the chaos threshold. At 
this point (where Ai = but X q ^ 0), the sensitivity function presents strong fluctuations 
with time and delimits the power-law growth of the upper bounds of a complex time 
dependence of the sensitivity to the initial conditions. These upper bounds allow us, on 
a log-log plot, to measure the slope oc t 1 ^ 1 ^^) from where the proper q = q* value 
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of the dynamical system under consideration can be estimated. This constitutes method 
I for determining q* value of a dynamical system at the edge of chaos (or at any point 
where Ai = 0) and it has already been successfully used for a variety of one-dimensional 
dissipative maps such as the standard logistic |18[, z- logistic f0| , circle PD| , ^-circular 



PT|| , single-site [22| and asymmetric logistic |23[] maps. At this stage, there is no a priori 



reason for relating this q index with that of the Tsallis entropy. It might well be that the 
corresponding entropy is the Renyi entropy since it also produces the power-law behaviour. 

The above concluding statement is also valid for method II of obtaining the q* value 
of a dynamical system. This method is based on the multifractal geometrical aspects of 
chaotic attractor at the edge of chaos. This geometry is characterized by the multifractal 
singularity spectrum f(a), which reflects the fractal dimension of the subset with singu- 
larity strenght a [0,0. The f{a) function is a down- ward parabola-like concave curve 
and at the end points of this curve, the singularity strenght is associated with the most 
concentrated (a min ) and the most rarefied (a max ) regions of the attractor. The scaling 
behaviour of these regions has been used to propose a new scaling relation as 

1 q O^rnin ^max 

which constitutes a completely different way of calculating the proper q* value of a given 
dynamical system ||20|| . It is evident from the previous works that, for all dissipative 
systems studied so far, the results of these two methods for the proper q* value coincide 
within a good precision. 

Finally, the connection between these q* values and the index q of the Tsallis entropy 
became evident after the introduction of method III for finding q* values . This method 
basically makes use of a specific generalization of Kolmogorov-Sinai (KS) entropy K\. For 
a chaotic dynamical system, the rate of loss of information can be characterized by this 
entropy and it is defined as the increase of the BG entropy per unit time. Since the Pesin 
equality states that K\ — Ai if Ai > and K\ = otherwise, it is clear that the KS 



entropy is deeply related to the Lyapunov exponents. Moreover, the KS entropy could be 



defined through 



SUt) 

K\ = lim lim lim , (6) 

t^oo W—*oo N^oo t 



where t is the number of discrete time steps in case of maps, W is the number of regions 
in the partition of the phase space and iV is the number of initial conditions that are 
evolving with time. Analogously, for the marginal generalized version of the KS 



entropy K q has been introduced recently |J| by replacing the BG entropy with the Tsallis 
entropy, namely, 

K Q = lim lim lim (7) 

Consistently, the Pesin equality is expected to become K q = X q if X q > and K q = 
otherwise. Using these ideas, the method III |25| , |2"6|| is based on the conjecture that (i) a 



unique value of q* exists such that K q is finite for q = q*, vanishes for q > q* and diverges 
for q < q*; (ii) this q* value coincides with that coming from methods I and II. Latora et 
al. have examined numerically the standard logistic map 

x t+ i = 1 - a x\ , (8) 

(where < a < 2 and — 1 < Xt < 1) both in the chaotic region and at the chaos threshold 
||25|| . In the chaotic region (for example, a = 2 case), they numerically verified that K q 
vanishes (diverges) for any value of q > 1 (q < 1), being finite only for q = 1 (see Fig.l 



of [[25|]), which implies that the proper q* value is unity as expected. On the other hand, 
at the chaos threshold, the same structure has also been observed, but this time with an 
important exception: K q vanishes (diverges) for any value of q > q* (q < q*), being finite 
only for q = q* = 0.24... (see Fig. 4 of ||25|| ). This value coincides with the value of the 
proper q* coming from other two methods. This structure has been verified not only for 



the standard logistic map but also a variety of other map families as well p2] , |23| , |26 . 

Now we are ready to proceed with our main purpose in this effort. We conjecture 
here that if the Tsallis and Renyi entropies are both the correct choices for a system with 



power-law behaviour, then the Renyi entropy should also give us the proper q* values 
whenever we use this entropic form instead of the Tsallis entropy in method III. Before 
demonstrating our results, let us describe the numerical procedure: Firstly, we partition 
the phase space into W equal cells, then we choose one of these cells and select N initial 
conditions all inside the chosen cell. As time evolves, these N points spread within the 
phase space and it gives a set {Ni(t)} with YhLi Ni(t) = N, Vt, which consequently yields 
a set of probabilities from where one can calculate the entropy. In order to compare our 
results to those of Latora et al., we numerically check the Renyi entropy case for the 
standard logistic map both in the chaotic region and at the chaos threshold. The results 
are illustrated in Fig.l. Surprisingly, the expected structure has not been observed. For 
the chaotic region (Fig. la), it is seen that, for q = 1 case, the linear entropy increase is 
realized but very strangely other values of q also give the same result! This shows that 
Renyi entropy is a much less sensitive function as compared to Tsallis entropy with respect 
to changes in the value of q. Similarly, at the chaos threshold (Fig. lb), Renyi entropy does 
not allow us to distinguish the correct q* value for which a linear rate is expected. Note 
that rate of increase of Renyi entropy can be linear if we plot the entropy with respect 
to ln(t) variable, which follows due to the relation between Tsallis and Renyi entropy, 
(compare (|l|) and (@)). This is another reason for unsuitability of the Renyi entropy; 
for Tsallis entropy we get the result of strong chaos (BG entropy proportional to t) in 
the q — > 1 limit, but for Renyi entropy, q — > 1 implies that BG entropy is proportional 
to \n(t). These observations strongly suggest that the Renyi entropy is not the suitable 
entropic form for these dynamical systems which exhibit power-law behaviour at their 
marginal points (where Ai = 0). Although the results which led us to make this claim 
seem rather convincing, it would be no doubt convenient to seek further evidences which 
can strengthen this claim. This is what we shall try to do in the remainder of the paper 
by making use of the generalized bit cumulants. 
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III. GENERALIZED BIT CUMULANTS 



Bit statistics is a tool to describe the complicated probability distributions, such as 
generated by chaotic systems [jlTp . In this framework, the first cumulant is the BG entropy 
itself, the second cumulant gives the variance (a measure of fluctuations) of the 'classical' 
bit number hi = —In pi, and so on. Particularly, the second bit cumulant which is a 
generalization of specific heat, can be applied in the context of equilibrium and non- 
equilibrium phase transitions J27|| . The classical bit cumulants can be generalized [^] 
within TT formalism. They were applied to symmetric logistic and ^-logistic family of 
maps given by 

x t+ i = 1 - a\x t \ z , (9) 

where the inflexion parameter z>l,0<a<2 and — 1 < x < 1 and t = 0,1,2,.... 
Applications have also been found for asymmetric family of logistic maps ||29|| , given by 





- a\x t \ Zl , 


if 


X t 


>A 




- a\x t \ z ' 2 , 


if 


x t 





x t+ i = { , (10) 



where Zj > 1. We briefly summarise the motivation and also the information obtained 
from these applications, as we intend to apply this approach to elucidate the difference in 
the use of Renyi and Tsallis entropies. 

The generalized second bit cumulant, which gives variance of the Tsallis bit number 



a 



(T) 



(l-g) 2 

For q — y 1, ( |lT|) approaches the standard bit variance, 



11) 



c 2 = j2(^Pi) 2 Pi-(Y,p^ P i) 2 . (12) 



It was observed in [28| that the ratio C 2 jGi evaluated in the chaotic region of the map 
(|), gives a scaling factor referred to as 'slope' in this paper (see Figs.), whose value 
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depends on values of q as well as the inflexion parameter z. Naturally, the slope tends 
to unity for q — > 1. Less trivially, it was observed that the slope tends to unity also for 
z — ► oo. In other words, increasing z value within the map-family, has the same effect 
on the quantifier 'slope' as the effect of taking to unity, the q parameter entering in its 
definition. Thus the otherwise free parameter q in (JTT|) behaves analogous to the proper q* 
value, which shows a monotonic decrease with increasing z values [Uj. This observation 
was used to conjecture the behaviour of proper index q* versus (z\, z<i) pairs |29|] in maps 



(|i0|) at chaos threshold. The actual behaviour obtained from finding q* values for these 



maps by using the previously introduced methods [23|, agree with the conjecture of |2 



This implies that generalized second bit cumulant (|TT|) can be consistently applied in such 
an analysis. 

Can we have generalization of bit-cumulants based on the Renyi entropy ? If such 
a possibility exists, then these new quantities may be used to study the implications on 
the above mentioned maps, and a comparison be made of the predictions derived from 
Tsallis based bit-cumulants and Renyi based ones. This task of a new generalization is 
not a straightforward one, in view of the fact that there is no generalized bit number 
corresponding to the Renyi entropy, i.e., the latter cannot be written as usual mean 
over some bit numbers, unlike the cases of BG and Tsallis entropies. In fact, Renyi 
entropy can be expressed only as a kind of nonlinear average |3(J. Recently, however, 



a step was made in this direction by one of the authors |31| . Specifically, the discrete 
derivative operator can be used to generate two types of bit cumulants, say type A and 
type B. Type A cumulants are Tsallis type, of the kind formulated in [28|. These are 



nonextensive quantities with respect to independent subsystems and preserve the standard 
relations between bit-moments and cumulants, as do the cumulants and moments of a 
probability distribution for a certain random variable. Type B 'cumulants' are however 
extensive quantites but do not preserve the standard relations between bit-moments and 
cumulants. Still, the approach allows us to clearly distinguish the separate origins of 
Tsallis and Renyi entropies, based on the use of discrete derivative. Particularly, the first 



bit cumulant of type B is the Renyi entropy, the second bit cumulant is another positive 
valued g-generalization of the standard bit variance, 

1 



d R) 



1 ~ <l ~ q) 2 



In Efe) 29 " 1 - 21n 



(13) 



It also recovers the form QT2] ) in the limit q — > 1. Thus it is interesting to see the 
application of (|13|) analogous to that of (|TlD for the 1-d dissipative maps. We expect that 
any difference in the predictions based on ( |l"3"D as compared to those of (|TTD , also reflects 
the separate implications for Renyi and Tsallis entropies for such systems. 

In the following, we compare the results obtained from using the generalized second 
bit-cumulants ( pTTj) and ([IB]) . In Fig. 2a, the results for symmetric maps @ are given. The 
slope which implies the ratio G\ /C 2 , shows a non-monotonic behaviour with increasing 
z values. This can be contrasted with the monotonic decrease of the ratio C 2 /C 2 , as 
shown in Fig. 2b. Similarly, for asymmetric maps (|T0|), we see separate trends as is clear 
by comparing Figs. 3a and 3b. 

This makes us to conclude that while the q parameter in the definition of generalized 

(T) 

cumulant C 2 behaves like the q* parameter inferred from the sensitivity to initial con- 
ditions at chaos threshold (method I), the q parameter in the definition C 2 serves no 
such connection. Thus this generalization of the standard cumulants, which are related 
to Renyi entropy, are inappropriate for such studies. This clearly strengthens our claim 
that the Renyi entropy is not the appropriate entropic form for such dynamical systems 
exhibiting power-law behaviour. 



IV. RATE OF ENTROPY INCREASE REVISITED 

So far, we have argued and shown that although it is expected to be applicable to sys- 
tems exhibiting power law behaviour, Renyi entropy is not a suitable measure to quantify 
the rate of entropy increase at points of power-law sensitivity in 1-d maps. However, 
Tsallis entropy qualifies as the appropriate measure for this purpose. A natural question 
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follows: Is Tsallis entropy the unique measure in this regard ? In this section, we propose 
that Tsallis entropy is not the unique measure of entropy which can give a linear rate of 
increase at chaos threshold. We exemplify below by proposing an alternative nonextensive 
entropy which can also yield a linear rate of increase. However, our conjecture is that the 
Tsallis entropy seems to be the unique entropy which yields the same q* as obtained from 
the modified sensitivity function (|J) and multifractal spectrum (f|), i.e., methods I and II 
to determine q*. To understand this role of the Tsallis entropy, we have to appreciate the 
implication of Pesin's equality. 

First, let us discuss the case of strong exponential sensitivity Ai > 0. In (||) assuming 
equiprobability, Pi = 1/M(t), where M(t) are the number of microstates or cells occupied 
at time t, we have 

Ki = lim lim lim -In M(t). (14) 

i^oo W->oo N^oo t 

Standard Pesin's equality K\ = Ai allows us to relate the above expression with exponen- 
tial sensitivity function. As a result, we get 

m = M(t). (15) 

In other words, for strong exponential sensitivity the number of states increase linearly 

with the distance between trajectories for all times t > 0. 

When Ai = 0, the use of modified sensitivity function (||) is suggested. Then we have 

Aq > 0. Using the definition (|7|) of K q and assuming equiprobability, we get 

t lM(t) 1 - q -l . . 

K a = lim lim lim . (16) 

* t^oo W-^oo N^oo t 1 — q 

Now postulating the generalized Pesin equality for the proper q* value of the entropic 
index (which yields the linear rate of entropy increase), K q * = X q *, we get from (|j) and 
(|I~6|), the same relation as (p~5|) even for points of power-law sensitivity. Thus the use of the 
Tsallis entropy in the definition of K q ensures that the linear relation (|T5D is preserved. 
Finally, this automatically implies that the critical q* value obtained in methods I and III 
are identical. 
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Now consider an alternate entropic form 

l_y.„( 1 + ln <?) 

S « = J=\ (9 G ^ + ) > ( 17 ) 

which has the same nonextensive property as the Tsallis entropy. It approximates the 
Tsallis entropy for values of q very near to one, and thus goes to BG entropy in the limit 
q — > 1. For this entropy, we can also obtain a linear rate of increase at chaos threshold, 
for certain critical value of q = q c . But in this case, q c ^ q*, where q* is calculated 
from methods I and II. Instead, we have 1 + In q c = q* . A more thorough understanding 
of generalized Pesin's equality is required to relate such other entropies with the issue 
of power-law sensitivity. At the moment, we only state that there are other entropic 
forms which can give a linear rate of increase, but the entropic form compatible with the 
generalized sensitivty function (£|), in the sense that methods I and III should yield same 
q* values, is the Tsallis entropy. 



V. CONCLUSIONS 

In this study, the appropriate entropic form for systems exhibiting power-law behaviour 
has been tried to be determined. To accomplish this task, two different issues have been 
analysed. One of these issues is the sensitivity to initial conditions and the entropy 
increase rates, the other one is the generalized bit cumulants. From the investigation of 
both issues, for the dynamical systems at marginal points (where the standard Lyapunov 
exponent vanishes), it is verified that the suitable entropic form is the Tsallis entropy. 
This could be considered as the first step towards the answer of the intriguing question: 
What is the correct entropic form for systems exhibiting power-law behaviour ? 

The reasons for the success of the Tsallis entropy, we believe, are two-fold: i) Tsallis 
entropy as function of time t, reproduces the correct q — > 1 limit of BG entropy propor- 
tional to t, ii) Tsallis entropy is a much more sensitive function than Renyi entropy with 
respect to changes in q value, which helps to determine the proper q* value in method III. 
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A crucial property of Tsallis entropy is its nonextensivity. It is not certain whether this 



property plays significant role in such studies. As has been argued in [5^1, a conditional 
entropy which is nonextensive for generic values of q, becomes extensive for the proper 
value of q = q*. It seems that, for such kind of systems with power-law behaviour, what 
is important is NOT to be extensive for all q values, but rather, to be extensive for only 
a special value of q (namely q*), the value that gives the correct entropy for the system 
under investigation. 

Finally, it is worth mentioning that we also show that the Tsallis entropy is not the 
unique entropic form which gives a linear entropy increase rate, but on the other hand, 
it seems to be the unique one which provides q* values consistent with those obtained 
from the other two methods (i.e., from the sensitivity function and from the multifractal 
scaling relation). 
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Figure Captions 

Figure 1 - Time evolution of the Renyi entropy at the chaotic region (a) and at the 
chaos threshold (b) for various q values. Inset of (b): The behaviour of the Renyi entropy 
for lnt . 

Figure 2 - The behaviour of the slope as a function of the map inflexion parameter 
z for the Renyi (a) and Tsallis (b) cases. 

Figure 3 - The behaviour of the slope as a function of the map inflexion parameter 
pairs (zi,z 2 ) for the Renyi (a) and Tsallis (b) cases. 



16 



Fig la 




time 



Fig lb 



Oh 
O 

u 

-J— > 

•l-H 
^ 1 







• 4=0.24 


(b) 


■ 4=0.5 




- 4=0.7 , V 

. ■i l A 1 

a=1.40115518909 t iJl 11 






















/ 












▲ 






1 " 


s 4 

I I I II 






1 10 



i i i i i i i r 
5 10 15 20 25 30 35 40 



time 



Fig 2a 



1.00 



0.99 



0.98 



0.97 



0.96 



0.95 



0.94 



0.93 




z 



Fig 2b 



1.9 



1.8 



1.7 



1.6 



1.5 



1.4 



1.3 



1.2 



Tsallis 



q=0.98 



(b) 



q=0.97 




~i i r 
2 3 4 



n i i i i i 
5 6 7 8 9 10 11 



z 



Fig 3 a 




z 2 z 2 



Fig 3b 



1.9 



1.8 



1.7 



1.6 



1.5 



1.4 



1.3 



1.2 




z 2 z 2 



